Multistage Classiication by Cascaded Classiiers

|We propose a new method of classiication built as a cascade of a distributed learner and a local learner. The distributed learner generalizes to learn the \rule" and the local learner learns the \exceptions" not covered by the \rule." We show how such a system can be trained using cross-validation. We use a multi-layer perceptron with sigmoidal hidden units as the rule-learner and a k-nearest neighbor classiier as the exception-learner. Cascading is a better approach than voting where multiple learners are used for all cases; the extra computation and memory required for the second learner is unnecessary if we are suuciently sure that the rst one's response is correct. The cascade algorithm sig-niicantly outperforms the individual methods and voting on three optical and pen-based handwritten digit recognition tasks when comparison is based on three criteria; generalization success, learning speed, and number of free parameters.